Skip to content

feat: add new query types - DescribeRouteTables/DescribeSubnets/DescribeSecurityGroupRules - #11

Merged
ytsarev merged 2 commits into
mainfrom
feat/describe-ec2
Sep 7, 2026
Merged

feat: add new query types - DescribeRouteTables/DescribeSubnets/DescribeSecurityGroupRules#11
ytsarev merged 2 commits into
mainfrom
feat/describe-ec2

Conversation

@jakubramut

Copy link
Copy Markdown
Contributor

Description of your changes

Why

Two identifiers can't be discovered through any existing query type:

  • aws_main_route_table_association - its external name is the main association ID, which only ec2:DescribeRouteTables returns. The AWS::EC2::RouteTable CloudFormation schema has just RouteTableId, Tags, VpcId.
  • Live subnets and route tables. GetResources keeps returning deleted resources: measured 16 entries for 10 real resources, the 6 stale ones carrying the same identifying tag as their live replacements.

ListResources is not a workaround. AWS::EC2::Subnet, AWS::EC2::SecurityGroup and AWS::EC2::SubnetRouteTableAssociation all publish no handlerSchema on their list handler, so Cloud Control cannot filter them server-side at all: it enumerates the type account-wide and hydrates each item, and in a shared account one foreign resource failing hydration aborts the caller's whole composition. A direct describe with a server-side filter reads only what was asked for.

What

queryType: DescribeEc2, with parameters.operation selecting RouteTables, SecurityGroupRules or Subnets. Reuses the existing toEC2Filters, so filters takes native EC2 names (vpc-id, group-id).

Tests

go build, go test ./..., golangci-lint run (v2.8.0, the CI pin) all clean.

Verified against real EC2 in eu-central-1 (VPC + subnets + route table + associations + security group with tcp, all-protocol and group-referencing rules), then torn down.
Live runs found two defects, both fixed here:

  • securityGroupRuleArn was dropped from the rule projection; real DescribeSecurityGroupRules returns it on every rule. Now projected.
  • the fixture assumed an ipProtocol: -1 rule omits ports. Real EC2 sends -1/-1. Fixture corrected, with a third rule keeping the genuinely-absent case covered.

Live runs also confirmed referencedGroupId unwraps from the nested ReferencedGroupInfo, rule tags project, and subnetId is populated for subnet associations and empty for the main one.

@ytsarev ytsarev left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good PR. Clear motivation, and the Cloud Control failure mode is documented accurately. Build, tests and lint clean locally.

Three blockers inline: an invalid vpc-id filter pinned by a test, no must-filter guard, and the example missing from make render. Smaller notes alongside them.

Design question: registry() already maps a CRD-enum-validated queryType to a handler. Routing on a free-form parameters.operation means operation: routeTables passes admission and fails at reconcile, and duplicates the operation list in four places.

Registering DescribeRouteTables/DescribeSubnets/DescribeSecurityGroupRules directly would get schema validation for free and match the existing convention. If avoiding enum churn is the deliberate tradeoff, worth saying so in the description, and deriving the error string from the dispatch table.

Comment thread aws.go Outdated
Comment thread example/README.md Outdated
Comment thread aws.go
Comment thread aws.go
Comment thread README.md Outdated
Comment thread input/v1beta1/input.go Outdated
Comment thread aws_test.go Outdated
Comment thread aws_test.go Outdated
@jakubramut jakubramut changed the title feat: add a DescribeEc2 query type feat: add new query types - DescribeRouteTables/DescribeSubnets/DescribeSecurityGroupRules Sep 7, 2026
@jakubramut

jakubramut commented Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

@ytsarev, thanks for the review. All three blockers were real and are fixed; replies inline on each thread.

On the design question: taken as proposed. DescribeEc2 + parameters.operation is gone, replaced by DescribeRouteTables / DescribeSubnets / DescribeSecurityGroupRules in the registry, CRD enum regenerated, and the runtime default: branch deleted outright.

Your argument is stronger than the schema-validation framing suggests, given this function returns a fatal result on any error: a typo like operation: routeTables did not degrade, it aborted the whole composition at reconcile. In the consumer's history one such mistake burned 27 minutes for zero resources. Enum validation moves that to admission time. It is an API change, which is exactly why it belongs here rather than after a release.

@ytsarev

ytsarev commented Sep 7, 2026

Copy link
Copy Markdown
Member

Checked c658ab1 out locally. go build ./..., go test ./... and golangci-lint run (v2.11.4) all clean, and CI is green.

All eight threads addressed, seven resolved. The design change went further than I suggested: folding the shared precondition into ec2Client with a per-operation hint means the error names the filters that operation accepts, rather than one generic list. Better than what I proposed.

Two things I checked specifically:

  • No codegen drift. The enum in package/input/aws.fn.crossplane.io_inputs.yaml matches input.go exactly across all 11 values. Worth noting CI has no drift check, so this holds by discipline rather than enforcement. Might be worth a go generate diff step in a follow-up.
  • Filter guard has real teeth. TestDescribeEc2FilterGuard asserts on message content rather than just err != nil, and covers all three query types.

I also liked the call on absent values: documenting the two policies and explaining why omitting beats faking is a better outcome than the shape change I was angling for.

Two cosmetic leftovers, neither blocking:

  1. README.md:56, left in the open thread above.
  2. Test names still say DescribeEc2 (aws_test.go:493 section header, plus TestDescribeEc2Dispatches, TestDescribeEc2FilterGuard, TestDescribeEc2RouteTablesPaginates, TestDescribeEc2RegionGuard, TestDescribeEc2Subnets, TestDescribeEc2SecurityGroupRules) for a query type that no longer exists.

Nothing outstanding from my side beyond those.

Comment thread README.md Outdated
Comment thread aws_test.go Outdated
@jakubramut

Copy link
Copy Markdown
Contributor Author

Both cosmetic leftovers are fixed; replies on the two threads. grep -rn DescribeEc2 across the repo is now empty.

On the codegen drift check: confirmed independently, go generate ./input produces no diff. But it should not go in this repo. .github/workflows/ci.yml here is synced from upbound/sa-up (see 737a83a), so a step added locally would be reverted on the next sync. It needs to land in the shared workflow template instead, where every function repo gets it rather than just this one. Happy to open that upstream.

Gates after the rename: go build, go vet, go test -race, gofmt, golangci-lint v2.8.0 all clean.

@ytsarev ytsarev left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified at b0f89d1: go build ./..., go test ./... and golangci-lint run all clean, grep -rn DescribeEc2 empty across the tree, and the CRD enum matches input.go.

Good iteration. Splitting into enum-validated query types, and folding the precondition into ec2Client with a per-operation hint, came out better than what I suggested. Same for documenting the absent-value policy instead of faking zeros.

The only thing left is the codegen drift check we discussed, which is a follow-up rather than a blocker.

@ytsarev
ytsarev merged commit 85c3211 into main Sep 7, 2026
6 checks passed
@ytsarev
ytsarev deleted the feat/describe-ec2 branch September 7, 2026 14:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants